references/v3-catalog/src/trigger/concurrency.ts (7)

20-51: Check ID references and consider parameter-based configuration.
You changed the task ID to "test-concurrency-controller" and introduced a childDelay parameter. Ensure that any external or downstream references expecting the old ID or missing the new parameter are updated. Also, the hard-coded 2-second wait (line 44) may be better served by a configurable parameter or environment variable if timing is critical.

---

58-71: Optional error handling for multi-step calls.
The parent task triggers testConcurrencyChild.triggerAndWait, but there is no explicit error handling. If a child task fails, it might be beneficial to handle or log the error to avoid silent failures.

---

153-164: Consider consistent return format.
You’re returning { completedAt: new Date() }. Other parts of the code sometimes use ISO strings. A unified format (e.g., .toISOString()) can help with downstream parsing.

---

166-183: Returning a standard timestamp format might help.
Like the controller task, this parent task returns { completedAt: new Date() }. If you want consistent logs across tasks, consider .toISOString().

---

201-223: Make propagation behavior explicit.
The logic conditionally triggers the grandchild if propagate is undefined or truthy. Consider defaulting propagate to true to make the intended behavior clearer.

-export const testChildTaskPriorityChild = task({
...
-run: async ({ delay = 5000, propagate }: { delay: number; propagate?: boolean }) => {
+run: async ({ delay = 5000, propagate = true }: { delay: number; propagate?: boolean }) => {
...
}

---

242-259: Optionally unify timestamp format.
Here again, the returned completedAt is a raw date object. If you need consistent time formats across your tasks, consider new Date().toISOString().

---

265-289: Hierarchical tasks are well-structured.
Your parentTask, subtask, and subsubtask effectively demonstrate a chained sequence within myQueue. Consider optional error handling or try/catch blocks if any step can fail unexpectedly.

apps/webapp/app/v3/marqs/envPriorityDequeuingStrategy.server.ts (4)

11-13: Use composition over direct assignment for the _delegate property.
To facilitate clearer dependency management, consider injecting _delegate through a parameter in the constructor and then referencing it directly, rather than reassigning in the constructor body. This will simplify testing and ensure consistency in initialization.

---

15-25: Ensure consistency in naming.
The method name distributeFairQueuesFromParentQueue is descriptive, but double-check it aligns with the broader naming scheme in your codebase (e.g., “distributeQueuesFromParentQueue” or “getFairQueuesFromParentQueue”). Consistent naming helps maintain clarity across similar classes and strategies.

---

58-63: Potential optimization by combining grouping and priority comparison.
Currently, each queue is grouped, then sorted, and subsequently all results are sorted again by priority. Consider a single pass approach to reduce complexity and overhead.

---

90-94: Graceful default for missing priority.
Returning zero for a missing priority is reasonable. If you anticipate rare scenarios where zero might be a valid priority and you must differentiate between “no priority” vs. “lowest priority,” consider using negative values or a separate property to distinguish.

references/test-tasks/src/utils.ts (4)

6-32: Use exponential backoff when polling run status.
The loop might issue a request every second, which could burden the system under heavy usage. An exponential or incremental backoff strategy can reduce load while waiting for long-running tasks.

---

34-45: Clarify error messages when environment variables are missing.
The existing error messages (“TRIGGER_API_URL is not set”, “TRIGGER_ACCESS_TOKEN is not set”) are simple but might be insufficient if multiple environment variables are expected. Consider providing guidance on how to configure them.

---

46-70: Improve error handling for concurrency limit updates.
When the response is not OK, the error message references response.statusText. Ensure that deeper error details (e.g., body content) are logged or captured if available. This could help in diagnosing issues on the server side.

---

84-109: Add caching or memoization for environment stats if calls are frequent.
Fetching environment stats in a tight loop or at high frequency can incur overhead. If the data changes infrequently, consider caching responses for short durations.

apps/webapp/app/routes/admin.api.v1.environments.$environmentId.ts (3)

68-70: Validate additional search parameters if needed.
The SearchParamsSchema currently allows only the optional queue parameter. If you anticipate future query params, consider preparing a more flexible schema or a separate object for them.

---

72-93: Leverage consistent authentication checks.
The loader and the action functions share similar authentication logic. If possible, abstract the authentication into a reusable helper to avoid code duplication and ensure uniform security handling across endpoints.

---

110-145: Augment concurrency data with historical context or external references.
The concurrency snapshot is valuable, but if you anticipate debugging more complex concurrency issues (e.g., capacity planning), consider returning historical usage or referencing external metrics. This can help identify trends over time.

apps/webapp/app/components/admin/debugRun.tsx (3)

34-43: Enhance dialog accessibility.
Consider adding appropriate aria-* attributes for the DialogContent (e.g. aria-labelledby) to ensure screen readers correctly reference the dialog header or label. This can improve overall accessibility.

---

45-67: Handle potential fetcher errors more robustly.
While there's a fallback message ("Failed to get run debug data"), it might be helpful to log or display specific error information provided by fetcher in case of a network or server error.

---

69-79: Avoid repeated string prefix logic.
const keys = new MarQSShortKeyProducer("marqs:") and const withPrefix = (key: string) => \marqs:${key}`are slightly redundant. You might centralize the logic withinMarQSShortKeyProducer` to avoid string concatenation in multiple places.

apps/webapp/app/v3/marqs/fairDequeuingStrategy.server.ts (2)

222-245: Potential performance optimization in #orderQueuesByEnvs.
The method constructs and filters arrays multiple times. If performance becomes a concern with large snapshots, consider a single pass approach or an in-place transformation.

---

420-461: Validate weighted selection logic in #selectTopEnvs.
The approach to average queue ages and use weighted random selection is interesting, but ensure it aligns with your fairness goals. Large differences in average ages might starve certain environments if random seeds align unfavorably for them.

references/test-tasks/src/trigger/test-reserve-concurrency-system.ts (4)

9-54: Structure long test steps with sub-tests.
describeReserveConcurrencySystem runs multiple complex steps in sequence. To improve readability and debugging, consider splitting it into sub-tests or smaller tasks (e.g., testRetryPriority, testResumePriority) invoked in a top-level orchestrator.

---

125-173: Ensure freeze/resume logic is fully tested.
testResumePriority asserts that resumed runs complete before newly queued runs. Consider also validating partial progress or re-entrant states if a run is paused multiple times, ensuring robust coverage of the resume mechanism.

---

367-501: Watch out for potential queue deadlocks with deep recursion.
recursiveTask calls itself via triggerAndWait or batchTriggerAndWait. If depth is large and concurrency is limited, nested calls could exhaust capacity and stall. Consider imposing a maximum recursion depth or concurrency-based backoff.

---

531-538: Enhance batch error reporting.
unwrapBatchResult only reports the first failing run’s error. If multiple runs fail, it may mask subsequent errors. Aggregating or logging all errors could improve debugging.

apps/webapp/test/fairDequeuingStrategy.test.ts (8)

79-118: Test coverage for reserve concurrency is clear.
These lines add a welcome scenario for environments with reserve concurrency. The logic looks consistent with the new concurrency approach. Consider adding edge cases (e.g., reserveConcurrency at 0 or negative) to validate error conditions.

---

221-225: Flattening result for speed test.
Using flattenResults helps keep this test logic clear. The performance check is a strong addition. Consider verifying a tolerance rather than a strict “9 times faster” ratio, to avoid random CI noise.

---

249-250: Performance ratio check.
The ratio-based performance assertion might occasionally be flaky if environment load varies. If that becomes an issue, consider a more robust approach (e.g., mocking or smaller ratio).

---

794-794: setupConcurrency usage.
You’re setting { currentConcurrency: 0, limit: 5 } for each environment, which is a good baseline for concurrency. Possibly consider an environment with already-exceeded concurrency to test rejection.

---

831-831: Accumulate environment selections.
Incrementing selection counts is a simple approach for measuring distribution. Ensure you reset these counts if you expand the test scenarios.

---

835-835: Console debug for analysis.
Logging selected environment counts helps short-term debugging but can clutter test output. Consider conditional logs or a debug flag if logs become noisy.

---

844-845: Sorting environment frequencies.
Slicing or sorting the map is a handy approach. Make sure large environment sets handle sorting well.

---

848-850: Top-two environments verification.
Aligning with highest average age is correct. Keep an eye on test flakiness if your distribution algorithm seeds differently in the future.

apps/webapp/app/v3/services/triggerTask.server.ts (1)

567-596: Inline run enqueue with error handling.
Enqueueing runs only if they are “PENDING” is logical. The immediate rollback to SYSTEM_FAILURE if enqueueResult.ok is false prevents half-baked states. Nicely handled.

apps/webapp/app/v3/services/batchTriggerV3.server.ts (1)

459-469: Directly aborting batch on error.
Switching to "ABORTED" status with completedAt avoids repeated re-queues. This is cleaner than retrying indefinitely. If partial retries are needed, consider capturing partial progress before abort.

apps/webapp/app/v3/marqs/index.server.ts (1)

489-489: Remove unnecessary continue.

This continue statement can be safely removed without changing the loop's behavior.

-              continue;

🧰 Tools

🪛 Biome (1.9.4)

[error] 489-489: Unnecessary continue statement

Unsafe fix: Delete the unnecessary continue statement

(lint/correctness/noUnnecessaryContinue)

apps/webapp/app/v3/services/enqueueRun.server.ts (2)

31-31: Address the TODO comment about deadlock handling.

The TODO comment indicates that deadlock handling needs to be implemented for reserveConcurrency.

Would you like me to help implement the deadlock handling logic or create an issue to track this task?

---

32-48: Consider adding retry logic for enqueue operation.

The marqs.enqueueMessage call could benefit from retry logic to handle transient failures.

Here's a suggested implementation using exponential backoff:

-  const wasEnqueued = await marqs.enqueueMessage(
+  const wasEnqueued = await retry(
+    async () => await marqs.enqueueMessage(
       env,
       run.queue,
       run.id,
       {
         type: "EXECUTE",
         taskIdentifier: run.taskIdentifier,
         projectId: env.projectId,
         environmentId: env.id,
         environmentType: env.type,
       },
       run.concurrencyKey ?? undefined,
       run.queueTimestamp ?? undefined,
       dependentRun
         ? { messageId: dependentRun.id, recursiveQueue: dependentRun.queue === run.queue }
         : undefined
+    ),
+    {
+      retries: 3,
+      minTimeout: 1000,
+      factor: 2,
+    }
   );

apps/webapp/app/routes/resources.taskruns.$runParam.debug.ts (2)

16-38: Consider adding error handling for database query timeouts.

The database query could benefit from timeout handling to prevent long-running queries.

+  const QUERY_TIMEOUT_MS = 5000;
+  const timeoutPromise = new Promise((_, reject) => 
+    setTimeout(() => reject(new Error('Query timeout')), QUERY_TIMEOUT_MS)
+  );
+
+  const run = await Promise.race([
   $replica.taskRun.findFirst({
     where: { friendlyId: runParam, project: { organization: { members: { some: { userId } } } } },
     select: {
       // ... existing select fields
     },
   }),
+    timeoutPromise
+  ]).catch(error => {
+    if (error.message === 'Query timeout') {
+      throw new Response("Query timeout", { status: 504 });
+    }
+    throw error;
+  });

---

44-62: Consider batching concurrency metric requests.

The multiple sequential calls to marqs for fetching concurrency metrics could be optimized by batching them together.

+  const [
+    queueConcurrencyLimit,
+    envConcurrencyLimit,
+    queueCurrentConcurrency,
+    envCurrentConcurrency,
+    queueReserveConcurrency,
+    envReserveConcurrency
+  ] = await Promise.all([
+    marqs.getQueueConcurrencyLimit(run.runtimeEnvironment, run.queue),
+    marqs.getEnvConcurrencyLimit(run.runtimeEnvironment),
+    marqs.currentConcurrencyOfQueue(run.runtimeEnvironment, run.queue, run.concurrencyKey ?? undefined),
+    marqs.currentConcurrencyOfEnvironment(run.runtimeEnvironment),
+    marqs.reserveConcurrencyOfQueue(run.runtimeEnvironment, run.queue, run.concurrencyKey ?? undefined),
+    marqs.reserveConcurrencyOfEnvironment(run.runtimeEnvironment)
+  ]);

apps/webapp/app/components/runs/v3/BatchStatus.tsx (1)

62-88: Consider adding a11y attributes for status colors.

The status colors should include ARIA attributes for better accessibility.

 export function BatchStatusIcon({
   status,
   className,
+  ariaLabel,
 }: {
   status: BatchTaskRunStatus;
   className: string;
+  ariaLabel?: string;
 }) {
   switch (status) {
     case "PENDING":
-      return <Spinner className={cn(batchStatusColor(status), className)} />;
+      return <Spinner className={cn(batchStatusColor(status), className)} aria-label={ariaLabel ?? batchStatusTitle(status)} />;
     case "COMPLETED":
-      return <CheckCircleIcon className={cn(batchStatusColor(status), className)} />;
+      return <CheckCircleIcon className={cn(batchStatusColor(status), className)} aria-label={ariaLabel ?? batchStatusTitle(status)} />;
     case "ABORTED":
-      return <XCircleIcon className={cn(batchStatusColor(status), className)} />;
+      return <XCircleIcon className={cn(batchStatusColor(status), className)} aria-label={ariaLabel ?? batchStatusTitle(status)} />;

apps/webapp/app/v3/services/enqueueDelayedRun.server.ts (1)

37-67: Consider optimizing the nested includes in the query.

The deeply nested includes in the query could impact performance. Consider using a more targeted query structure.

-  include: {
-    dependency: {
-      include: {
-        dependentBatchRun: {
-          include: {
-            dependentTaskAttempt: {
-              include: {
-                taskRun: true,
-              },
-            },
-          },
-        },
-        dependentAttempt: {
-          include: {
-            taskRun: true,
-          },
-        },
-      },
-    },
-  },
+  select: {
+    dependency: {
+      select: {
+        dependentBatchRun: {
+          select: {
+            dependentTaskAttempt: {
+              select: {
+                taskRun: {
+                  select: {
+                    id: true,
+                    queue: true,
+                  },
+                },
+              },
+            },
+          },
+        },
+        dependentAttempt: {
+          select: {
+            taskRun: {
+              select: {
+                id: true,
+                queue: true,
+              },
+            },
+          },
+        },
+      },
+    },
+  },

apps/webapp/app/v3/marqs/types.ts (1)

110-113: Add documentation for the recursiveQueue flag.

Please add JSDoc comments explaining the purpose and behavior of the recursiveQueue flag in EnqueueMessageReserveConcurrencyOptions.

 export type EnqueueMessageReserveConcurrencyOptions = {
+  /** The unique identifier of the message to be enqueued */
   messageId: string;
+  /** Whether to recursively process the queue. Add more details about when this should be true/false */
   recursiveQueue: boolean;
 };

apps/webapp/test/utils/marqs.ts (1)

57-88: Add detailed documentation about the dummy jobs approach.

While the implementation is clean, it would be helpful to add more detailed documentation explaining why dummy jobs are used to simulate concurrency and reserved concurrency.

 /**
  * Sets up concurrency-related Redis keys for orgs and envs
+ * 
+ * @remarks
+ * This function uses dummy job IDs to simulate both current and reserved concurrency.
+ * Dummy jobs are used because... [explain the rationale]
+ * 
+ * @param options - The options for setting up concurrency
+ * @param options.redis - The Redis instance
+ * @param options.keyProducer - The key producer instance
+ * @param options.env - The environment configuration
+ * @param options.env.id - The environment ID
+ * @param options.env.currentConcurrency - The number of concurrent jobs to simulate
+ * @param options.env.limit - Optional concurrency limit
+ * @param options.env.reserveConcurrency - Optional number of reserved concurrent jobs
  */

apps/webapp/app/v3/services/resumeTaskDependency.server.ts (2)

53-72: Improve the TODO comment.

The current TODO comment is informal and lacks clarity. Please update it to better describe what needs to be done.

-      // TODO: use the new priority queue thingie
+      // TODO: Implement priority queue system for task dependency resumption
+      // This should handle message priorities using MarQSPriorityLevel

---

92-106: Document the timestamp fallback logic.

Please add a comment explaining why we fall back to createdAt when queueTimestamp is not available.

+      // Fall back to createdAt if queueTimestamp is not available
+      // This ensures consistent message ordering while maintaining backward compatibility
       await marqs?.replaceMessage(

apps/webapp/app/v3/marqs/marqsKeyProducer.ts (2)

23-312: Add JSDoc comments for better documentation.

Consider adding JSDoc comments to describe the purpose, parameters, and return values of public methods. This would improve code maintainability and developer experience.

Example for the queueKey method:

+/**
+ * Generates a queue key based on the provided parameters.
+ * @param envOrOrgId - The environment object or organization ID
+ * @param queueOrEnvId - The queue name or environment ID
+ * @param queueOrConcurrencyKey - The queue name or concurrency key
+ * @param concurrencyKeyOrPriority - Optional concurrency key or priority
+ * @param priority - Optional priority value
+ * @returns The generated queue key
+ */
 queueKey(
   envOrOrgId: MarQSKeyProducerEnv | string,
   queueOrEnvId: string,
   queueOrConcurrencyKey: string,
   concurrencyKeyOrPriority?: string | number,
   priority?: number
 ): string {

---

283-285: Add input validation for priority values.

The prioritySection method should validate that the priority value is a positive integer to prevent potential issues.

 private prioritySection(priority: number) {
+  if (!Number.isInteger(priority) || priority < 0) {
+    throw new Error(`Invalid priority value: ${priority}. Must be a positive integer.`);
+  }
   return `${constants.PRIORITY_PART}:${priority}`;
 }

apps/webapp/app/v3/services/createCheckpoint.server.ts (1)

306-306: Consider removing redundant undefined argument.

The explicit undefined argument is redundant as it's the default value when no argument is provided.

-            undefined,

apps/webapp/test/marqsKeyProducer.test.ts (3)

10-14: Consider extracting test data setup to a shared helper.

The test data setup for TestDelegate could be moved to a shared helper function to improve reusability and maintainability.

+class TestDelegate implements MarQSFairDequeueStrategy {
+  static createWithQueues(queues: EnvQueues[]): TestDelegate {
+    return new TestDelegate(queues);
+  }
+
   constructor(private queues: EnvQueues[]) {}

   async distributeFairQueuesFromParentQueue(): Promise<Array<EnvQueues>> {
     return this.queues;
   }
 }

---

158-162: Add test cases for invalid inputs.

Consider adding test cases for invalid queue strings to ensure robust error handling.

 it("should throw error for invalid queue string", () => {
   const invalidQueue = "invalid:queue:string";
   expect(() => producer.queueDescriptorFromQueue(invalidQueue)).toThrow("Invalid queue");
 });
+
+it("should throw error for empty queue string", () => {
+  expect(() => producer.queueDescriptorFromQueue("")).toThrow("Invalid queue");
+});
+
+it("should throw error for malformed queue string", () => {
+  expect(() => producer.queueDescriptorFromQueue("org:123:env:456:invalid:testQueue"))
+    .toThrow("Invalid queue");
+});

---

182-190: Improve test description for stable sort.

The test description could be more explicit about what is being tested.

-// Check that queue2 and queue4 (priority 1) maintain their relative order
-// and queue1, queue3, and queue5 (priority 0) maintain their relative order
+// Verify that:
+// 1. Higher priority queues (queue2, queue4 with priority 1) are sorted first
+// 2. Within the same priority level, the original order is preserved
+// 3. Non-priority queues (queue1, queue3, queue5) maintain their relative order at the end

apps/webapp/test/envPriorityDequeueingStrategy.test.ts (2)

19-40: Extract test data setup to a shared helper.

Consider extracting common test data setup to reduce duplication and improve maintainability.

+const createTestEnvQueues = (envId: string, queues: string[]): EnvQueues => ({
+  envId,
+  queues,
+});
+
+const createTestQueue = (
+  orgId: string,
+  envId: string,
+  queueName: string,
+  priority?: number,
+  concurrencyKey?: string
+): string => {
+  let queue = `org:${orgId}:env:${envId}:queue:${queueName}`;
+  if (concurrencyKey) queue += `:ck:${concurrencyKey}`;
+  if (priority !== undefined) queue += `:priority:${priority}`;
+  return queue;
+};

 const inputQueues: EnvQueues[] = [
   {
     envId: "env1",
     queues: [
-      "org:org1:env:env1:queue:queue1:priority:1",
-      "org:org1:env:env1:queue:queue2:priority:1",
-      "org:org1:env:env1:queue:queue3:priority:1",
+      createTestQueue("org1", "env1", "queue1", 1),
+      createTestQueue("org1", "env1", "queue2", 1),
+      createTestQueue("org1", "env1", "queue3", 1),
     ],
   },
 ];

---

193-225: Add test cases for additional edge cases.

Consider adding test cases for more edge cases to ensure robust behavior.

+it("should handle queues with extremely large negative priorities", async () => {
+  const inputQueues: EnvQueues[] = [
+    {
+      envId: "env1",
+      queues: [
+        "org:org1:env:env1:queue:queue1:priority:-9999999",
+        "org:org1:env:env1:queue:queue2:priority:1",
+      ],
+    },
+  ];
+
+  const delegate = new TestDelegate(inputQueues);
+  const strategy = new EnvPriorityDequeuingStrategy({
+    delegate,
+    keys: keyProducer,
+  });
+
+  const result = await strategy.distributeFairQueuesFromParentQueue("parentQueue", "consumer1");
+
+  expect(result[0].queues).toEqual([
+    "org:org1:env:env1:queue:queue2:priority:1",
+    "org:org1:env:env1:queue:queue1:priority:-9999999",
+  ]);
+});
+
+it("should handle queues with malformed priority values", async () => {
+  const inputQueues: EnvQueues[] = [
+    {
+      envId: "env1",
+      queues: [
+        "org:org1:env:env1:queue:queue1:priority:invalid",
+        "org:org1:env:env1:queue:queue2:priority:1",
+      ],
+    },
+  ];
+
+  const delegate = new TestDelegate(inputQueues);
+  const strategy = new EnvPriorityDequeuingStrategy({
+    delegate,
+    keys: keyProducer,
+  });
+
+  const result = await strategy.distributeFairQueuesFromParentQueue("parentQueue", "consumer1");
+
+  // Expect malformed priority to be treated as priority 0
+  expect(result[0].queues).toEqual([
+    "org:org1:env:env1:queue:queue2:priority:1",
+    "org:org1:env:env1:queue:queue1:priority:invalid",
+  ]);
+});

apps/webapp/app/env.server.ts (1)

360-360: Document the new environment limit configuration.

The new MARQS_MAXIMUM_ENV_COUNT configuration needs documentation to explain its purpose and impact.

+  /** Maximum number of environments allowed in the system.
+   * This replaces the previous organization-based limit to align with the new environment-centric architecture.
+   * @example 100
+   */
   MARQS_MAXIMUM_ENV_COUNT: z.coerce.number().int().optional(),

apps/webapp/app/v3/services/completeAttempt.server.ts (1)

467-482: Extract retry queue logic to a separate method.

The retry queue logic is complex and could be extracted to improve readability and maintainability.

+  private async retryViaQueue(
+    runId: string,
+    taskIdentifier: string,
+    checkpointEventId: string | undefined,
+    retryTimestamp: number
+  ) {
+    logger.debug("[CompleteAttemptService] Enqueuing retry attempt", { runId });
+
+    return marqs?.replaceMessage(
+      runId,
+      {
+        type: "EXECUTE",
+        taskIdentifier,
+        checkpointEventId: this.opts.supportsRetryCheckpoints ? checkpointEventId : undefined,
+        retryCheckpointsDisabled: !this.opts.supportsRetryCheckpoints,
+      },
+      retryTimestamp,
+      MarQSPriorityLevel.retry
+    );
+  }

   const retryViaQueue = () => {
-    logger.debug("[CompleteAttemptService] Enqueuing retry attempt", { runId: run.id });
-
-    // We have to replace a potential RESUME with EXECUTE to correctly retry the attempt
-    return marqs?.replaceMessage(
-      run.id,
-      {
-        type: "EXECUTE",
-        taskIdentifier: run.taskIdentifier,
-        checkpointEventId: this.opts.supportsRetryCheckpoints ? checkpointEventId : undefined,
-        retryCheckpointsDisabled: !this.opts.supportsRetryCheckpoints,
-      },
-      executionRetry.timestamp,
-      MarQSPriorityLevel.retry
-    );
+    return this.retryViaQueue(
+      run.id,
+      run.taskIdentifier,
+      checkpointEventId,
+      executionRetry.timestamp
+    );
   };

internal-packages/database/prisma/migrations/20250210164232_add_queue_timestamp_to_run/migration.sql (1)

4-6: Adding the queueTimestamp Column to TaskRun

A new column "queueTimestamp" (of type TIMESTAMP(3)) is added to the "TaskRun" table. This field will be useful for tracking when tasks are queued. Please verify whether this column should be nullable (as no default is provided) and confirm that downstream services and the Prisma schema are updated to handle a potential null value.

docs/queue-concurrency.mdx (4)

10-14: ‘Default Concurrency’ Section Added

The new “## Default concurrency” header and its accompanying text effectively describe that by default each task may run with unbounded concurrency (subject to the environment-level limits). The phrase “fill up the entire concurrency limit” is vivid; just be sure that the intended meaning is evident to new users.

---

257-257: Stylistic Suggestion on Intensifiers

The phrase “It’s very important to note…” appears at around line 257. Consider replacing “very important” with “important” to reduce overuse of intensifiers and improve style.

🧰 Tools

🪛 LanguageTool

[style] ~257-~257: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...e parent task slot is reoccupied. It's very important to note that we only release at-most X ...

(EN_WEAK_ADJECTIVE)

---

70-70: Typographical Correction Suggestion

A static analysis hint suggests inserting a comma for better readability on a sentence around line 70. Please review the sentence starting “When you trigger a task…” and consider adding a comma (e.g. after a leading clause) to improve clarity.

🧰 Tools

🪛 LanguageTool

[typographical] ~70-~70: Consider inserting a comma for improved readability.
Context: ...u trigger a run When you trigger a task you can override the concurrency limit. Thi...

(INITIAL_ADVP_COMMA)

---

219-219: Comma After “for example”

A LanguageTool hint flags that after “for example” a comma is usually expected. Adding this comma can improve the sentence flow in that portion of the text.

🧰 Tools

🪛 LanguageTool

[typographical] ~219-~219: After the expression ‘for example’ a comma is usually used.
Context: ...yload) => { //... }, }); ``` For example purposes, let's say the environment con...

(COMMA_FOR_EXAMPLE)

internal-packages/database/prisma/schema.prisma (1)

1734-1735: New queueTimestamp Field in TaskRun Model

A new optional field queueTimestamp DateTime? is added to the TaskRun model. This field will allow the system to record when a run is queued, which supports the new concurrency/reservation functionality. Ensure that application logic (and any queries/sorts) referencing task timing is updated accordingly.

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

apps/webapp/app/v3/marqs/index.server.ts (1)

486-515: 🛠️ Refactor suggestion

Add error handling for message acknowledgment failures.

The acknowledgeMessage method should handle potential Redis operation failures more gracefully.

Add error handling:

 public async acknowledgeMessage(messageId: string, reason: string = "unknown") {
   return this.#trace(
     "acknowledgeMessage",
     async (span) => {
       const message = await this.readMessage(messageId);

       if (!message) {
         logger.log(`[${this.name}].acknowledgeMessage() message not found`, {
           messageId,
           service: this.name,
           reason,
         });
         return;
       }

+      try {
         span.setAttributes({
           [SemanticAttributes.QUEUE]: message.queue,
           [SemanticAttributes.MESSAGE_ID]: message.messageId,
           [SemanticAttributes.CONCURRENCY_KEY]: message.concurrencyKey,
           [SemanticAttributes.PARENT_QUEUE]: message.parentQueue,
           ["marqs.reason"]: reason,
         });

         await this.options.visibilityTimeoutStrategy.cancelHeartbeat(messageId);

         await this.#callAcknowledgeMessage({
           parentQueue: message.parentQueue,
           messageQueue: message.queue,
           messageId,
         });

         await this.options.subscriber?.messageAcked(message);
+      } catch (error) {
+        logger.error(`Failed to acknowledge message`, {
+          messageId,
+          error,
+          service: this.name,
+          reason,
+        });
+        throw error;
+      }
     },

🧰 Tools

🪛 Biome (1.9.4)

[error] 489-489: Unnecessary continue statement

Unsafe fix: Delete the unnecessary continue statement

(lint/correctness/noUnnecessaryContinue)

references/v3-catalog/src/trigger/concurrency.ts (1)

153-258: 🛠️ Refactor suggestion

Use SDK's wait utility and add documentation for the test scenario.

1. Replace all instances of setTimeout with the SDK's wait.for() utility.
2. Consider adding documentation to explain the test scenario and task hierarchy.

+/**
+ * Test scenario for queue priority in task hierarchies:
+ * - Parent task triggers child task
+ * - Child task can trigger grandchild task based on propagate flag
+ * - All tasks use the same queue with concurrency limit of 1
+ * - Demonstrates priority handling for resuming/retrying tasks
+ */
 export const testChildTaskPriorityController = task({

The implementation effectively tests queue priority for task hierarchies, which aligns with the PR objectives.

apps/webapp/app/v3/services/resumeTaskDependency.server.ts (2)

3-3: Remove unused import.

The MarQS type is imported but never used in this file.

-import { MarQS, marqs, MarQSPriorityLevel } from "~/v3/marqs/index.server";
+import { marqs, MarQSPriorityLevel } from "~/v3/marqs/index.server";

---

53-53: Remove outdated TODO comment.

The TODO comment about using "the new priority queue thingie" is outdated since the priority queue implementation is already in place with MarQSPriorityLevel.resume.

-      // TODO: use the new priority queue thingie

apps/webapp/app/v3/marqs/envPriorityDequeuingStrategy.server.ts (1)

27-88: Consider adding JSDoc comments for the sorting logic.

The sorting implementation is robust, maintaining stability for equal priorities and properly handling queue groups. However, the complex logic would benefit from documentation explaining the sorting strategy.

Add JSDoc comments to explain the sorting strategy:

+/**
+ * Sorts queues within each environment based on priority.
+ * Maintains original order for queues with equal priority.
+ * @param envs Array of environment queues
+ * @returns Sorted array of environment queues
+ */
 #sortQueuesInEnvironmentsByPriority(envs: EnvQueues[]): EnvQueues[] {
   return envs.map((env) => {
     return this.#sortQueuesInEnvironmentByPriority(env);
   });
 }

+/**
+ * Sorts queues within an environment based on priority.
+ * Groups queues by base name and selects highest priority queue from each group.
+ * Maintains original order for queues with equal priority.
+ * @param env Environment queues to sort
+ * @returns Environment with sorted queues
+ */
 #sortQueuesInEnvironmentByPriority(env: EnvQueues): EnvQueues {

apps/webapp/app/v3/marqs/types.ts (1)

20-71: Consider grouping related methods in the interface.

The interface is comprehensive but could benefit from logical grouping of related methods using JSDoc comments.

Add method grouping comments:

 export interface MarQSKeyProducer {
+  // Queue-related key methods
   queueConcurrencyLimitKey(env: MarQSKeyProducerEnv, queue: string): string;
   
+  // Environment-related key methods
   envConcurrencyLimitKey(envId: string): string;
   envConcurrencyLimitKey(env: MarQSKeyProducerEnv): string;
   
+  // Message-related key methods
   messageKey(messageId: string): string;
   nackCounterKey(messageId: string): string;

apps/webapp/app/v3/marqs/marqsKeyProducer.ts (2)

220-260: Consider adding input validation for queueDescriptorFromQueue.

While the method handles missing components well, it should also validate the format of individual components.

Add validation for component formats:

 queueDescriptorFromQueue(queue: string): QueueDescriptor {
+  if (!queue || typeof queue !== 'string') {
+    throw new Error('Queue must be a non-empty string');
+  }
+
   const match = queue.match(QUEUE_REGEX);
   
   if (!match) {
     throw new Error(`Invalid queue: ${queue}, no queue name found`);
   }
   
   const [, queueName] = match;
+  
+  if (!queueName || queueName.length === 0) {
+    throw new Error(`Invalid queue: ${queue}, queue name cannot be empty`);
+  }

---

262-285: Consider adding JSDoc for private helper methods.

The private helper methods would benefit from documentation explaining their purpose and parameters.

Add JSDoc comments:

+/**
+ * Shortens an ID by taking its last 12 characters.
+ * @param id The full ID to shorten
+ * @returns The shortened ID
+ */
 private shortId(id: string) {
   return id.slice(-12);
 }

+/**
+ * Creates an environment key section.
+ * @param envId Environment ID
+ * @returns Formatted environment key section
+ */
 private envKeySection(envId: string) {
   return `${constants.ENV_PART}:${this.shortId(envId)}`;
 }

apps/webapp/test/marqsKeyProducer.test.ts (2)

131-162: Add more error case tests for queueDescriptorFromQueue.

While basic error cases are covered, consider adding tests for empty strings and invalid component formats.

Add additional test cases:

 describe("queueDescriptorFromQueue", () => {
+  it("should throw error for empty queue string", () => {
+    expect(() => producer.queueDescriptorFromQueue("")).toThrow("Queue must be a non-empty string");
+  });
+
+  it("should throw error for queue with empty name", () => {
+    expect(() => producer.queueDescriptorFromQueue("org:123:env:456:queue:")).toThrow(
+      "queue name cannot be empty"
+    );
+  });

---

176-222: Consider more descriptive test case names.

While the test cases are comprehensive, their names could be more descriptive of the specific scenarios being tested.

Improve test case names:

-it("should generate correct current concurrency key", () => {
+it("should generate current concurrency key with basic queue name", () => {

-it("should include concurrency key when provided", () => {
+it("should include concurrency key in current concurrency key when provided", () => {

-it("should remove the priority bit when provided", () => {
+it("should exclude priority from current concurrency key when priority is present", () => {

apps/webapp/app/v3/marqs/fairDequeuingStrategy.server.ts (3)

155-194: Consider adding error handling for edge cases in #shuffleQueuesByEnv.

The method could encounter issues if:

1. maxLimit calculation results in -Infinity (when envs array is empty)
2. Weighted calculations with very large numbers could cause precision issues

Consider adding these safeguards:

 const maxLimit = Math.max(...envs.map((envId) => snapshot.envs[envId].concurrency.limit));
+// Handle empty envs case
+if (!Number.isFinite(maxLimit) || maxLimit <= 0) {
+  return [];
+}

 const weightedEnvs: WeightedEnv[] = envs.map((envId) => {
   const env = snapshot.envs[envId];
   let weight = 1;

   if (biases.concurrencyLimitBias > 0) {
-    const normalizedLimit = env.concurrency.limit / maxLimit;
+    // Prevent division by zero and ensure normalized values
+    const normalizedLimit = maxLimit > 0 ? env.concurrency.limit / maxLimit : 0;
     weight *= 1 + Math.pow(normalizedLimit * biases.concurrencyLimitBias, 2);
   }

---

487-490: Remove unnecessary continue statement.

The continue statement at the end of the catch block is unnecessary as it's the last statement in the loop.

 } catch (error) {
   // Log error but continue trying other queues
   logger.warn(`[${this.name}] Failed to dequeue from queue ${messageQueue}`, { error });
-  continue;
 }

---

1799-1865: Consider extracting Redis configuration into a separate function.

The Redis configuration setup in getMarQSClient is quite lengthy and could be made more maintainable.

Consider refactoring like this:

function createRedisConfig() {
  if (!env.REDIS_HOST || !env.REDIS_PORT) {
    throw new Error(
      "Could not initialize Trigger.dev because process.env.REDIS_HOST and process.env.REDIS_PORT are required to be set."
    );
  }

  return {
    keyPrefix: KEY_PREFIX,
    port: env.REDIS_PORT,
    host: env.REDIS_HOST,
    username: env.REDIS_USERNAME,
    password: env.REDIS_PASSWORD,
    enableAutoPipelining: true,
    ...(env.REDIS_TLS_DISABLED === "true" ? {} : { tls: {} }),
  };
}

function getMarQSClient() {
  const redis = new Redis(createRedisConfig());
  const keysProducer = new MarQSShortKeyProducer(KEY_PREFIX);
  
  return new MarQS({
    // ... rest of the configuration
  });
}

apps/webapp/app/v3/marqs/index.server.ts (2)

1288-1670: Consider splitting Redis command definitions into a separate file.

The Redis command definitions section is quite large and could be better maintained in a dedicated file.

Consider creating a new file redisCommands.ts to house these definitions, improving maintainability and separation of concerns.

---

1107-1119: Consider reducing debug logging verbosity.

The debug logging in the Redis command calls is quite verbose and could impact performance in production.

Consider adding a verbosity level check:

+const shouldLogVerbose = process.env.NODE_ENV !== 'production' || this.options.verbose;

 logger.debug("Calling dequeueMessage", {
+  ...(shouldLogVerbose ? {
     messageQueue,
     parentQueue,
     queueConcurrencyLimitKey,
     envConcurrencyLimitKey,
     queueCurrentConcurrencyKey,
     queueReserveConcurrencyKey,
     envCurrentConcurrencyKey,
     envReserveConcurrencyKey,
     envQueueKey,
     service: this.name,
+  } : {
+    messageQueue,
+    service: this.name,
+  })
 });

references/v3-catalog/src/trigger/concurrency.ts (2)

19-56: Consider adding JSDoc documentation.

The task implementation looks good and aligns with testing the reserve concurrency system. Consider adding JSDoc documentation to explain the purpose of the parameters and the test scenario.

 export const testConcurrency = task({
+  /**
+   * Tests the reserve concurrency system by triggering multiple parent tasks
+   * that create child tasks, demonstrating queue priority and concurrency management.
+   * @param count - Number of tasks to trigger in batch
+   * @param delay - Delay for parent task execution
+   * @param childDelay - Delay for child task execution
+   */
   id: "test-concurrency-controller",

---

260-289: Document queue sharing and potential deadlock scenario.

The implementation demonstrates queue sharing between tasks, which is a key scenario mentioned in the PR objectives. However, this setup could potentially lead to a Recursive Queue Deadlock when tasks in the same queue wait for each other.

Consider adding documentation to:

1. Explain the purpose of sharing the same queue
2. Warn about potential deadlock scenarios
3. Describe how the reserve concurrency system handles this

 export const myQueue = queue({
+  /**
+   * Queue shared between parent, subtask, and subsubtask to demonstrate:
+   * 1. Queue priority for resuming/retrying tasks
+   * 2. Reserve concurrency system handling recursive waits
+   * Note: Tasks sharing the same queue with triggerAndWait pattern
+   * could lead to deadlock without proper reserve concurrency handling.
+   */
   name: "my-queue",
   concurrencyLimit: 1,
 });

apps/webapp/app/v3/services/enqueueRun.server.ts (1)

21-66: Consider adding error handling for edge cases.

The function handles the basic recursive wait deadlock case, but there might be other edge cases to consider:

1. What happens if marqs.enqueueMessage throws an error?
2. What happens if the environment or run objects are invalid?

Consider adding try-catch blocks and input validation:

 export async function enqueueRun({
   env,
   run,
   dependentRun,
 }: EnqueueRunOptions): Promise<EnqueueRunResult> {
+  // Validate input
+  if (!env || !run) {
+    return {
+      ok: false,
+      error: {
+        type: "INTERNAL_ERROR",
+        code: TaskRunErrorCodes.CONFIGURED_INCORRECTLY,
+        message: "Invalid environment or run object",
+      },
+    };
+  }
+
+  try {
     const wasEnqueued = await marqs.enqueueMessage(
       env,
       run.queue,
       run.id,
       {
         type: "EXECUTE",
         taskIdentifier: run.taskIdentifier,
         projectId: env.projectId,
         environmentId: env.id,
         environmentType: env.type,
       },
       run.concurrencyKey ?? undefined,
       run.queueTimestamp ?? undefined,
       dependentRun
         ? { messageId: dependentRun.id, recursiveQueue: dependentRun.queue === run.queue }
         : undefined
     );

     if (!wasEnqueued) {
       const error = {
         type: "INTERNAL_ERROR",
         code: TaskRunErrorCodes.RECURSIVE_WAIT_DEADLOCK,
         message: `This run will never execute because it was triggered recursively and the task has no remaining concurrency available`,
       } satisfies TaskRunError;

       return {
         ok: false,
         error,
       };
     }

     return {
       ok: true,
     };
+  } catch (error) {
+    return {
+      ok: false,
+      error: {
+        type: "INTERNAL_ERROR",
+        code: TaskRunErrorCodes.TASK_EXECUTION_FAILED,
+        message: `Failed to enqueue run: ${error instanceof Error ? error.message : String(error)}`,
+      },
+    };
+  }
 }

apps/webapp/app/v3/services/resumeBatchRun.server.ts (3)

191-210: Consider adding retry logic for message enqueuing.

The message enqueuing operation could fail due to transient issues. Consider implementing a retry mechanism with exponential backoff.

Consider wrapping the enqueue operation in a retry function:

async function retryEnqueueMessage(
  marqs: any,
  environment: any,
  queue: string,
  messageId: string,
  payload: any,
  concurrencyKey?: string,
  queueTimestamp?: Date,
  dependentRun?: any,
  priority?: MarQSPriorityLevel,
  maxRetries = 3
): Promise<boolean> {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await marqs.enqueueMessage(
        environment,
        queue,
        messageId,
        payload,
        concurrencyKey,
        queueTimestamp,
        dependentRun,
        priority
      );
    } catch (error) {
      if (attempt === maxRetries) throw error;
      await new Promise(resolve => setTimeout(resolve, Math.pow(2, attempt) * 100));
    }
  }
  return false;
}

---

255-273: Consider adding validation for timestamp conversion.

The timestamp conversion using getTime() could throw if the date is invalid.

Add validation before converting the timestamp:

-          (
-            dependentTaskAttempt.taskRun.queueTimestamp ?? dependentTaskAttempt.taskRun.createdAt
-          ).getTime(),
+          const timestamp = dependentTaskAttempt.taskRun.queueTimestamp ?? dependentTaskAttempt.taskRun.createdAt;
+          if (!(timestamp instanceof Date) || isNaN(timestamp.getTime())) {
+            logger.warn("Invalid timestamp for message replacement", {
+              timestamp,
+              dependentTaskAttemptId: dependentTaskAttempt.id
+            });
+            return new Date().getTime();
+          }
+          timestamp.getTime(),

---

191-210: Remove or update the informal TODO comment.

The TODO comment uses informal language ("thingie"). Either remove it since the priority queue implementation is already in place, or update it with a more professional description.

-        // TODO: use the new priority queue thingie

The enqueueMessage parameters are correctly updated with timestamp and priority level.

packages/core/src/v3/errors.ts (1)

916-931: Consider adding test coverage for error string conversion.

The new taskRunErrorToString function handles all error types but lacks test coverage.

Consider adding tests to verify the string conversion for each error type:

describe('taskRunErrorToString', () => {
  it('should format internal errors', () => {
    const error: TaskRunError = {
      type: "INTERNAL_ERROR",
      code: "RECURSIVE_WAIT_DEADLOCK",
      message: "Test message"
    };
    expect(taskRunErrorToString(error)).toBe("Internal error [RECURSIVE_WAIT_DEADLOCK]: Test message");
  });

  it('should format built-in errors', () => {
    const error: TaskRunError = {
      type: "BUILT_IN_ERROR",
      name: "TypeError",
      message: "Test error"
    };
    expect(taskRunErrorToString(error)).toBe("TypeError: Test error");
  });

  // Add more test cases for STRING_ERROR and CUSTOM_ERROR
});

apps/webapp/app/v3/services/triggerTask.server.ts (2)

569-595: Consider adding logging for enqueue failures.

The error handling for enqueue failures could benefit from detailed logging.

Add logging to track enqueue failures:

 if (!enqueueResult.ok) {
+  logger.error("Failed to enqueue run", {
+    runId: run.id,
+    friendlyId: run.friendlyId,
+    error: enqueueResult.error,
+    taskId: run.taskIdentifier,
+    queue: run.queue,
+    environment: {
+      id: environment.id,
+      type: environment.type
+    }
+  });
+
   await this._prisma.taskRun.update({
     where: { id: run.id },
     data: {

---

602-617: Consider adding transaction rollback on error.

When throwing a ServiceValidationError, consider rolling back any database changes.

Add transaction rollback:

 if (result?.error) {
+  await this._prisma.$transaction(async (tx) => {
+    // Rollback any changes made to the run
+    await tx.taskRun.delete({
+      where: { id: result.run.id }
+    });
+  });
+
   throw new ServiceValidationError(
     taskRunErrorToString(taskRunErrorEnhancer(result.error))
   );
 }

references/test-tasks/src/utils.ts (3)

6-32: Consider validating the returned run object and adding a progressive backoff strategy

1. If runs.retrieve(id) returns null or some unexpected shape, the code might silently error when accessing run.status. You may want to confirm the run is non-null before checking its status.
2. Instead of a fixed 1-second delay, a progressive or exponential backoff can reduce API load while waiting for long-running tasks.

---

34-70: Validate concurrency limit before making the request

Currently, updateEnvironmentConcurrencyLimit passes concurrencyLimit directly through. You could:

1. Check if concurrencyLimit is ≥ 1 (or whichever minimum applies).
2. Enforce an upper bound if relevant.
3. Use a small Zod schema to keep the logic aligned with your other validated requests.

---

82-109: Ensure consistency with environment variables and handle errors gracefully

getEnvironmentStats relies on TRIGGER_API_URL and TRIGGER_ACCESS_TOKEN but does not explicitly validate their presence. Consider:

1. Verifying these variables (similar to updateEnvironmentConcurrencyLimit).
2. Providing a clearer error message if they are undefined.

references/test-tasks/src/trigger/test-reserve-concurrency-system.ts (4)

9-54: Consider error-isolation or cleanup across multiple tests

describeReserveConcurrencySystem runs many sub-tests sequentially. If a sub-test fails partway, some concurrency changes or states may linger. You might:

• Implement a higher-level try/finally to reset concurrency states.
• Isolate each sub-test in separate contexts if that suits your testing approach.

---

367-411: Guard against infinite recursion where depth could accidentally be large

recursiveTask depends on depth to terminate. Consider adding a safe upper bound or a sanity check to prevent unbounded recursion if inputs are malformed.

---

464-494: Code duplication across resumeParentTask and genericParentTask

Both tasks share similar logic for triggering child tasks and using batch or single runs. Consider extracting common logic into a helper to adhere to DRY principles.

---

503-530: Similar logic to resumeParentTask could be factored out

genericParentTask duplicates much of the same code path. Centralizing shared logic might improve maintainability.

apps/webapp/app/routes/admin.api.v1.environments.$environmentId.ts (2)

68-70: Optional queue parameter schema is good but could be extended

You might:

1. Validate that queue matches a known format (if you have naming conventions).
2. Provide helpful error messages for invalid queue inputs.

---

72-145: Enhance loader error handling & null checks

1. The function calls marqs.getEnvConcurrencyLimit and related methods; consider verifying marqs is defined, and handle potential connection or runtime errors.
2. If concurrency lookups fail, providing more descriptive error info can help debug.

apps/webapp/app/components/admin/debugRun.tsx (1)

80-340: Consider refactoring repeated Property.Item components.

The code contains multiple Property.Item components with similar structure. Consider creating a utility function or map-based approach to generate these items dynamically:

+type DebugProperty = {
+  label: string;
+  value: string;
+  iconButton?: boolean;
+};
+
+function PropertyItem({ label, value, iconButton = true }: DebugProperty) {
+  return (
+    <Property.Item>
+      <Property.Label>{label}</Property.Label>
+      <Property.Value className="flex items-center gap-2">
+        <ClipboardField
+          value={value}
+          variant="tertiary/small"
+          iconButton={iconButton}
+        />
+      </Property.Value>
+    </Property.Item>
+  );
+}

-<Property.Item>
-  <Property.Label>Queue key</Property.Label>
-  <Property.Value className="flex items-center gap-2">
-    <ClipboardField
-      value={withPrefix(
-        keys.queueKey(run.runtimeEnvironment, run.queue, run.concurrencyKey ?? undefined)
-      )}
-      variant="tertiary/small"
-      iconButton
-    />
-  </Property.Value>
-</Property.Item>
+<PropertyItem
+  label="Queue key"
+  value={withPrefix(
+    keys.queueKey(run.runtimeEnvironment, run.queue, run.concurrencyKey ?? undefined)
+  )}
+/>

This refactoring would:

1. Reduce code duplication
2. Make the code more maintainable
3. Reduce the risk of typos

docs/queue-concurrency.mdx (2)

70-71: Add commas after introductory phrases.

For improved readability, add commas after introductory phrases:

-When you trigger a task you can override the concurrency limit.
+When you trigger a task, you can override the concurrency limit.

-For example purposes let's say the environment concurrency limit is 1.
+For example purposes, let's say the environment concurrency limit is 1.

Also applies to: 219-220

🧰 Tools

🪛 LanguageTool

[typographical] ~70-~70: Consider inserting a comma for improved readability.
Context: ...u trigger a run When you trigger a task you can override the concurrency limit. Thi...

(INITIAL_ADVP_COMMA)

---

257-258: Consider rephrasing for stronger emphasis.

Instead of using "very important", consider a stronger phrasing:

-It's very important to note that we only release at-most X slots
+It is crucial to note that we only release at-most X slots

🧰 Tools

🪛 LanguageTool

[style] ~257-~257: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...e parent task slot is reoccupied. It's very important to note that we only release at-most X ...

(EN_WEAK_ADJECTIVE)

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro